Analysis of biochemical analytes using six sigma metrics with two analyzers at an Indian lab setting

A zero defects goal was implemented in the clinical laboratory settings using a six-sigma model. Daily Internal Quality Control (IQC) and external quality control data from April-September 2023 was extracted to calculate the sigma metrics of 21 biochemical analytes based on Total Error Allowable (TEa), % bias and co-efficient of variation percent (CV%). A retrospective comparative study was conducted in the department of Clinical Biochemistry at Kanva Diagnostic Services Pvt. Ltd, Bengaluru, India. The analytical performance of the 21 biochemical analytes was tested on Cobas 6000 and C311 analyzers. Quality Goal Index (QGI) and root cause analysis was calculated to infer the reason for the deviation of six sigma. Method decision charts were plotted to show the comparison of the problem analytes on both the analyzers. On Cobas 6000 at level 1 IQC, out of 21 analytes, 10 analytes showed σ>6 and 10 analytes showed σ 3-6 and on C311, 15 analytes which showed σ>6 and 6 analytes that showed σ 3-6. On Cobas 6000 at level 2 IQC, out of 21 analytes, 12 analytes showed σ>6 and 8 analytes showed σ 3-6 and on C311 17 analytes showed σ>6 and 4 analytes showed σ 3-6. Creatinine failed to meet minimal sigma performance at both levels of IQC on Cobas 6000.


Background:
Total Quality Management System in the diagnostic laboratory aims at the proper collection, analysis, and conveyance of precise and prompt reports to the right patient [1].Clinical laboratory results play a significant role in decisions related to treatment of patients [2].The laboratory results if incorrect leads to serious complications like incorrect and delayed diagnosis and treatment [3][4].The Total Testing Process (TTP) entails pre-analytical, analytical and post-analytical processes [5][6].Analytical process is a dynamic area where maximum errors occurs leading to erroneous reports.Medical laboratories should strive to produce precise reproducible results as clinicians rely on these results for diagnosis, monitoring, and prognostication of patients [7][8].Sigma (σ) metrics, a bench mark used in the field of quality management and quality control, particularly in healthcare and laboratory settings, to assess and monitor the performance and accuracy of diagnostic tests and measurement processes [9].Bill Smith of Motorola Corporation invented Six Sigma process [10] and Nevalainen et al. first applied the six sigma model, in medical laboratories [11].These metrics help the clinical laboratories establish quality goals, ensure that their processes are meeting the required standards and make necessary corrections to maintain and enhance the quality of the patients' reports [12][13].The critical goal of sigma metrics is to implement risk management in the laboratory and to safeguard the patients [14].As a predictor of risk, sigma metrics is a statistical measure of the capability of a process to produce results within predefined specifications or limits [16][17].It's a way to assess how well a process is performing in terms of its ability to consistently produce output within acceptable quality boundaries.The exact number of errors in the analytical phase can be quantified only by Sigma metric and not by internal and external quality control data [18].Therefore, it is of interest to compare the sigma metrics of 21 biochemical analytes on Cobas 6000 and Cobas C311, evaluate the root causes and take corrective action to improve the performance of the analytes with poor sigma metrics.

Materials and methods:
A retrospective comparative study was conducted in the department of Clinical Biochemistry at Kanva Diagnostic Services Pvt.Ltd, Bengaluru.It is a standalone NABL accredited lab (MC-3756) which abides with the NABL guidelines and provides diagnostic services to around 400 outpatients daily.The equipment used for analysis was integrated modular analyzer Cobas 6000 and Cobas C311 (Roche Diagnostics, Mannheim, Germany).The data was extracted from consecutive runs of assay IQC samples for the 21 biochemical analytes over duration of six months from April to September 2023.The QC material used in the laboratory was a third party QC provided by BioRad which was received in lyophilized form and reconstituted by trained technical personnel.The IQC protocol was scheduled according to the NABL guidelines.Both physiological (Level 1) and pathological levels (Level 2) of QC were run daily before analyzing the patients' samples.The IQC data was monitored daily and the Levy-Jennings charts were interpreted using the standard Westgard rules (13S/22S/R4S/10x).
The laboratory has enrolled in monthly EQAS program provided by BioRad.A stringent root-cause analysis was implemented followed by the needed corrective action for any deviations in IQC and EQAS results.Since the defects in the analytical performance cannot be assessed by IQC and EQAS results alone and hence, sigma metrics is required to quantify the exact number of defects in the testing process.The obtained sigma metric is inversely proportional to the quantitative defects.

Statistical analysis
[1] Mean, SD and CV% were calculated for each analytes from the monthly IQC data.[2] TEa values of the study analytes were taken from Clinical Laboratory Improvements Amendment (CLIA) 2024 and from Biological Variation (BV) database by Dr Carmen Ricos and colleagues available at www.westgard.com[19].[3] The sigma metrics was calculated for all analytes using the above variables as mentioned below: a. CV%: Standard-deviation/Mean x 100 b.Bias%: Lab EQAS result-Peer group mean / Peer group mean c.Sigma Metric(σ) = TEa (%) -Bias(%)/CV% [4] Sigma metric of > 6 is world class performance, σ value of 3-6 is good performance and σ value of <3 indicates poor performance of the test.[5] QGI: % Bias/1.5 x CV% [6] QGI<0.8 = Imprecision, QGI: 0.8-1.2= Imprecision and Inaccuracy and QGI >1.2 = Inaccuracy [20].[7] Statistical software: The obtained data was entered in Microsoft Excel Version 16 and the histograms were plotted.[8] The normalized sigma method decision charts were extracted from the website https://www.westgard.com/normalized-opspecscalculator.htm.Parameters such as TEa, bias% and CV% were inputted and the graph was plotted with bias% on yaxis and imprecision on x-axis.Sigma metric zones are presented on the Sigma method decision charts i.e., the zone closest to the graph's origin 'World class performance' is 6σ zone, followed by the 5σ 'Excellent' zone, 4σ 'Good' zone, 3σ 'Marginal' zone, 2σ 'Poor' zone and the remaining portion of the chart is marked as unacceptable.

Results:
The current study evaluated the sigma metrics for 21 biochemical analytes run on Cobas 6000 and Cobas C311.The comparison of CV% of level 1 IQC for the biochemical on Cobas 6000 and Cobas C311 from the month of April to September 2023 are tabulated in Table 1.The comparison CV% of level 2 IQC for the biochemical analytes on Cobas 6000 and Cobas C311 from the month of April to September 2023 are emphasized in Table 2.The comparison the bias % for the biochemical analytes on Cobas 6000 and Cobas C311 from the month of April to September 2023 are displayed Table 3.The comparison of the Sigma metrics and QGI for biochemical analytes from the month of April to September 2023 are shown in Table 4.The performance of the 21 biochemical analytes on Sigma metrics scale are categorized into three levels i.e., >6,3-6 and <3 as summarized in Table 5. QGI for creatinine for level 1 IQC was 0.25 and for level 2 IQC was 0.24, which indicated imprecision in the QC values.

Discussion:
In the current study, the analytical performance of 21 biochemical analytes was compared on two automated analyzers, Cobas 6000 and Cobas C311.The sigma metrics was effectively evaluated for every analyte based on the IQC and EQAS data obtained from April 2023-September 2023.In clinical laboratories since the reliability of test reports relies on accuracy and precision, QGI was calculated for the analyte with σ<3 to reveal the accuracy and precision.The CV% measures variability and random error and Bias% which indicate accuracy and systemic errors in the testing process.TEa targets were derived from CLIA'2024 and BV data.
The graphic description on the working of the sigma metrics equation is depicted in Figure 1.On Cobas C311 at level 1 IQC, out of 21 analytes, 15 analytes which showed world class performance (σ>6) were Albumin, ALP, SGOT, Bilirubin Direct, Calcium, GGT, LDL, Phosphorus, Protein Total, Triglycerides, Uric acid, Cholesterol, HDL, Potassium and Chloride.The 6 analytes that showed good performance (σ3-6) were SGPT, Bilirubin Total, Creatinine, Glucose, Urea and Sodium as shown in Figure 3.All analytes showed good performance.Creatinine failed to meet minimal sigma performance at both levels of IQC on Cobas 6000.QGI calculated and root cause analysis for Creatinine showed imprecision.The corrective action to be adopted was an additional QC rule 41s apart from 13S /22S/R4S /10x and frequent calibration of Creatinine.A study in Beijing hospital was conducted under national creatinine trueness verification scheme.They used two different concentration levels of fresh frozen serum for evaluation of Creatinine measurement on automated analyzer Roche.The results showed that there was a requisite for 7-45.1% of the laboratories to improve their measurement procedures for enzymatic method.11.5-73 % of the laboratories must try to improve the trueness for Jaffe's method.3.1-5.3% of the laboratories ought to emphasize on both precision and trueness.The results of this study revealed poor performance for creatinine which is analogous to our study [22].A retrospective study conducted in Turkey, calculated six sigma values for 21 routine biochemistry parameters using Cobas c702.A three-month IQC data was taken for the calculation of CV%.The results of this study were in consensus with the results of our study with respect to creatinine i.e., according to CLIA goals the sigma metrics was <3 which showed imprecision at level 1IQC [24].
Contradictory results were reported in a retrospective study followed by prospective study conducted in clinical Biochemistry department in JSS Medical College, Mysuru.The IQC data for 31 analytes was collected from Cobas 6000 and e411 retrospectively followed by prospective study for analytes which showed σ<2.In contrast to our study, Creatinine showed world class performance (σ = 6.39) in level 1 internal QC and good performance in level 2 A retrospective observational study in Indonesia showed that sigma metrics calculated for 11 biochemical analytes on Cobas C311 showed good performance for Creatinine in accordance with the findings of the current study.However, urea showed unacceptable performance which was not in agreement with the findings of our study [27].

Conclusion:
The current study evaluated the performance of 21 biochemical analytes by sigma metrics on two automated analyzers Cobas 600 and C311.The study focused on to spot the parameter which deviated from the six sigma scale.The only biochemical analyte that showed poor performance on Cobas 6000 at both levels of IQC was Creatinine.The problem-solving strategy for the imprecision

Figure 2 :
Figure 2: Sigma metrics for biochemical analytes on Cobas 6000 level 1 IQC in descending order.

Figure 3 :
Figure 3: Sigma metrics for biochemical analytes on Cobas C311 level 1 IQC in descending order.

Figure 5 :
Figure 5: Sigma metrics for biochemical analytes on Cobas C311 level 2 IQC in descending order.

Figure 6
demonstrates the method decision charts plotted for the performance of Creatinine on Cobas 6000 and Cobas C311.The factors such as reconstitution of controls and calibrators, stability of controls and calibrators, equipment maintenance, scheduled preventive maintenance, quality of water used in analysis, on board stability of reagents, inadequate sample aspiration, improper mixing, contamination in sample cuvettes and training of the personnel were all taken care.Similar results were reported in a retrospective observational study conducted in clinical Biochemistry laboratory in KR hospital, Mysuru.Sigma metrics was calculated for renal function tests and electrolyte parameters which were analyzed on Cobas 6000.The study reported poor performance with σ<3 for Creatinine in level 1 and 2 IQC [21].An observational study conducted in Ethiopian Public Health Institute (EPHI) clinical chemistry reference laboratory tested 18 biochemical analytes on Cobas 6000.The results showed consensus with the results of the current study about Creatinine for both the levels of control.Unlike the findings of our study, low sigma metrics were reported for Urea and Chloride [3].

Figure 6 :
Figure 6: Sigma method decision chart showing Inaccuracy (bias%) is on y-axis and Imprecision (CV%) is on x-axis.A: Sigma metric 4.5 for level 1 QC for Creatinine on Cobas C311;B: Sigma metric 4.0 for level 2 QC for Creatinine on Cobas C311; C: Sigma metric 2.5 for level 1 QC for Creatinine on Cobas 6000; D: Sigma metric 2.3 for level 2 QC for Creatinine on Cobas 6000.A retrospective study in Clinical Laboratory of Hunan Provincial People's Hospital, China was performed using Beckman Coulter AU5800 analyzer and compared two level IQC sigma values of 19 biochemical analytes.Out of which 10 analytes showed good performance on sigma metric scale and 9 analytes showed a sigma value of <4.QGI analysis and RCA further indicated inaccuracy and imprecision.These findings were in consensus with the results of our study which showed imprecision for Creatinine at both levels IQC[23].

IQC (σ = 5 .
34), Sodium and Potassium showed poor performance (σ<2) [25].Contradicting with the results of the current study, a prospective study conducted in King Abdulaziz Specialist Hospital, Sakaka, evaluated the sigma metrics for 25 biochemical parameters on Cobas 6000.The performance of Creatinine (σ=4.66 at level 1 and σ=5.06 at level 2 IQC) and Sodium, Potassium and Chloride (σ<3) were not in agreement with the results of the current study [26].